Adapt validation to sonata examples #81
Conversation
Codecov Report
@@ Coverage Diff @@
## master #81 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 18 18
Lines 1329 1342 +13
=========================================
+ Hits 1329 1342 +13
Continue to review full report at Codecov.
|
asanin-epfl
force-pushed
the
adapt-validation-to-sonata-examples
branch
from
d2c389e to
d703211
Compare
bluepysnap/circuit_validation.py
Outdated
| """Transforms hdf5 population group to pandas DataFrame. | ||
|
|
||
| Args: | ||
| group: HDF5 group of edges or nodes population |
There was a problem hiding this comment.
I am pretty sure we cannot do that for edges.
With a "normal" edge file (few hundred of millions of edges to multiple billions of edges), the data cannot fit in memory.
Did you try with a big circuit ?
Maybe we should keep that for nodes only at the moment ?
There was a problem hiding this comment.
Ah ok I saw that you are using it only for the nodes.
Is it something you plan on using on edges too ?
There was a problem hiding this comment.
I thought to use it for edges but now I'm thinking. It will definitely fail on few hundred of millions of edges. I will reimplement it only for nodes.
| types = pd.read_csv(types_file, sep=r'\s+') | ||
| types.rename(columns={types.columns[0]: 'type_id'}, inplace=True) | ||
| return pd.merge(df, types, on='type_id', how='left') |
There was a problem hiding this comment.
that s cool it makes a first impl of the csv !
There was a problem hiding this comment.
Now we can validate Allen circuits as they heavily rely on this .csv.
|
That s really cool. Thanks for making this. Do we validate the AIBS circuit with these changes ? |
|
Yes. I did. Here is the log: |
|
So it spots errors that s great ! |
|
Thanks for the review! |
No description provided.